This invention relates to a image processing apparatus and an image processing method by which images taken with a digital still camera (which are hereinafter referred to as DSC images) or images recorded on a film or other images can be processed into appropriate finished photographic prints and the like. The invention particularly relates to an image processing apparatus and an image processing method that can yield finished photographic prints by compressing the dynamic range of the input image to produce a desired output image without causing halo or any other artifacts.
A variety of image processing techniques have conventionally been applied so that DSC images taken with a digital still camera (hereinafter abbreviated as DSC) or film images are processed to have an appropriate finish.
For example, when printing is done from a medium such as a film that has a wide dynamic range of shooting brightness to paper (photographic paper) that has a limited range of reproducible densities, a variety of image processing techniques are employed to ensure that the images recorded on the film are processed to produce appropriate finished photographic prints (refer to JP 3568279 B).
An image processing technique called “hyper tone processing” is also practiced to process DSC images or film images into appropriate finished photographic prints.
In hyper tone processing, input image data is separated into a lower-frequency component and a higher-frequency component and the lower-frequency component is subjected to gradation expansion, with further input image data being added to the gradation-expanded image data so as to produce an output image that remains as well-modulated as the input image and which yet has an expanded gradation. Hyper tone processing has the advantage that when printing is done from a medium such as a film that has a wide dynamic range of shooting brightness to paper (photographic paper) that has a limited range of reproducible densities, the loss of highlights and shadows can be reduced.
FIG. 6A is a block diagram showing a conventional image processing apparatus that performs hyper tone processing and FIG. 6B is a graph showing a conversion table in a lookup table (LUT) used in the conventional image processing apparatus shown in FIG. 6A.
As shown in FIG. 6A, the conventional image processing apparatus 110 that performs hyper tone processing has a light/dark signal generating unit 112, a blurred image generating unit 114, a lookup table (hereinafter abbreviated to LUT) 116, an image analyzing unit 117, and an adder 118. The input image data has been obtained with a scanner that reads the film having an image of the subject recorded thereon, and the prescanned data has been obtained by reading at a lower resolution than for the input image data.
The light/dark signal generating unit 112 generates binary light/dark image data (Y) from the image data for each of the colors R, G and B in the input image data (SR, SG, SB). The light/dark image data (Y) may be generated by calculating a third of the average taken from the input image data (SR, SG, SB) for the three colors R, G and B. In other words, the light/dark image data is generated as Y=(SR+SG+SB)/3.
The blurred image generating unit 114 elicits the lower-frequency component (Y_b) of the light/dark image data generated by the light/dark signal generating unit 112 (said component is hereinafter referred to as blurred image data (Y_b)) and this may be done by using a smoothing filter.
In the LUT 116, the blurred image data (Y_b) generated by the blurred image generating unit 114 is subjected to image processing so as to generate compression processing image data that is to be used in compressing the dynamic range of the input image data. A conversion table is set in the LUT 116 and as FIG. 6B shows, it is represented by a straight line 120 having a constant slope. By means of the LUT 116, the densities of the light areas of an image are added whereas those of the dark areas are subtracted. The central density of the image of the principal subject or the like remains constant.
The image analyzing unit 117 performs image analysis on the prescanned data and it changes the settings of the conversion table in the LUT 116 according to a particular scene.
The adder 118 adds up the input image data (SR, SG, SB) and the blurred image data (Y_b) obtained by processing in the light/dark signal generating unit 112, the blurred image generating unit 114, and the LUT 116. This generates output image data (OR, OG, OB) having a dynamic range compressed in comparison with the principal image data.
The output image data (OR, OG, OB) and the input image data (SR, SG, SB) can be correlated by the following mathematical expressions (1)-(3), in which LUT(Y_b) represents the entry of the blurred image data into the conversion table.OR=SR+LUT(Y—b)  (1)OG=SG+LUT(Y—b)  (2)OB=SB+LUT(Y—b)  (3)
In the conventional procedure of hyper tone processing, the input image data and the blurred image data obtained by processing that input image data in the light/dark signal generating unit 112, the blurred image generating unit 114 and the LUT 116 are added together, whereupon the dynamic range of the input image data is compressed nonlinearly to yield the result shown in FIG. 7, where input image data 130a that has a medium to higher frequency component 132a on the high density side and medium to higher frequency components 134a on the low density side has been converted to output image data 130b in which a medium to higher frequency component 132b on the high density side and medium to higher frequency components 134b on the low density side are maintained to lie within the print reproduction region. The output image data 130b thus obtained is appropriate not only in the dynamic range but also in the gradations and densities of the light and dark areas. As a result, photographic prints are produced that reproduce images of high quality.
An image processing apparatus that employs the above-described procedure of hyper tone processing has been proposed (refer to JP 3568279 B).
JP 3568279 B discloses an image reproducing method by which a digital image signal (digital image data) that represents a color image is reproduced as a visible image. In this image reproducing method in JP 3568279 B, the digital image signal is converted to a light/dark signal (light/dark image data) and on the basis of that light/dark signal, a blurred image signal (blurred image data) that represents a blurred image in the color image is generated. Then, subtraction is effected between the digital image signal and a signal for the pixel that corresponds to the blurred image signal, thereby generating a difference signal. A specified image processing procedure is performed on that difference signal to generate a processed image signal. Using this processed image signal, one can prepare photographic prints in which the image has a rather weakened overall contrast whereas the local contrast represented by the higher-frequency component is the same as in the original color image.
In JP 3568279 B, the blurred image signal which represents a blurred image in the color image is generated using a low-pass filter (LPF), so in edge portions that involve significant changes in density, the density changes are not as abrupt as in the original image but are small enough to cause so-called “dull” edge portions. If such a blurred image signal is used to generate the processed image signal, either an undershoot or an overshoot occurs in the edge portions. As a result, the finished prints that are finally obtained might have halo or other artifacts in the edge portions.
Under these circumstances, various methods have so far been proposed with a view to suppressing halo and other artifacts (refer to JP 2001-298619 A, JP 9-214766 A, and JP 2003-8898 A).
JP 2001-298619 A discloses an image processing apparatus having a plurality of low-pass filters and a LUT computing unit that generates a composite blurred image signal.
In the image processing apparatus of JP 2001-298619 A, an image signal for the original images is processed by the plurality of low-pass filters to generate a plurality of blurred image signals that represent a blurred image in the original image, and the generated blurred image signals are assembled in the LUT computing unit to generate a single composite blurred image signal. On the basis of this composite blurred image signal, the image signal for the original image is processed to compress the dynamic range of the latter. As a result, even if the original image is one of high contrast or wide dynamic range that has been taken against light or with the aid of an electronic flash, or even if it contains a relatively flat region such as a low-contrast area, it not only remains sufficiently well-modulated but what is more, one can reduce a pseudo-contour that will occur when the step of compressing the dynamic range is intensely applied. The image processing apparatus of JP 2001-298619 A is so adapted that in an image region that contains edge portions, the composite blurred image signal is adjusted to have a value close enough to a blurred image signal for a small degree of blur.
The apparatus of JP 2001-298619 A may be adapted to have the design shown in FIG. 8, where a blurred image generating unit 114 comprises a first IIR (infinite impulse response) filter 140, a second IIR filter 142, and a blurred image assembling unit 144.
The first IIR filter 140 and the second IIR filter 142 are each a kind of low-pass filter that is adapted to generate a blurred image signal. Suppose here that the first IIR filter 140 produces a blurred image signal for a large degree of blur whereas the second IIR filter 142 produces a blurred image signal for a small degree of blur. The blurred image assembling unit 144 is adapted to generate a composite blurred image signal.
Thus, in the image processing apparatus of JP 2001-298619 A, the two IIR filters 140 and 142 cooperate to generate a single composite blurred image signal.
JP 9-214766 A discloses an image processing apparatus that uses a median filter assembly, or a kind of edge preserving smoothing filter, as means of generating a blurred image signal.
In JP 9-214766 A, the median filter assembly comprises a plurality of median filters of different sizes, and the level of intermediate values is selected in accordance with the distribution of digital signals that generate the blurred image signal. In JP 9-214766 A, by using the median filter assembly, one can suppress the occurrence of overshoot in large edge portions.
JP 2003-8898 A discloses an image processing apparatus for converting an input image to an image having a relatively smaller dynamic range. This apparatus comprises smoothing means by which split segments of the input image are respectively smoothed to generate a plurality of smoothed images having different degrees of smoothness, edge intensity calculating means for calculating the edge intensity on the basis of the plurality of smoothed images, assembling means for assembling the plurality of smoothed images on the basis of the calculated edge intensity, coefficient calculating means by which the coefficients for converting the individual pixel values of the input image are calculated on the basis of the composite smoothed image generated by assembling the plurality of smoothed images, and pixel value converting means for converting the individual pixel values of the input image by the calculated coefficients.
In JP 2003-8898 A, filtering and downsampling of the input image are each performed more than once to generate a plurality of smoothed images, which are then assembled together; in this way, the input image is processed by an equivalently large scale of filtering (smoothing) and, as a result, the amounts of computing that are required by the individual steps of filtering and downsampling are sufficiently reduced that less computing needs to be performed for smoothing than when a single large filter is employed. In addition, the edge intensity is calculated on the basis of the plurality of smoothed images and on the basis of this edge intensity, the plurality of smoothed images are assembled together; hence, the downsampling operation causes no adverse effect on the accuracy of edge detection. This offers the advantage that even if illumination is effected under different conditions, their boundaries can be appropriately extracted while requiring smaller amounts of computing and this contributes to realizing a subjectively preferred compression of the dynamic range.
A plurality of epsilon filters are used as the smoothing means in JP 2003-8898 A. They are also a kind of edge preserving smoothing filters.
As described above, if the density range of certain input image data is broader than what can be reproduced on a particular medium such as photographic paper, it has been possible in the conventional technique to compress the selectively extracted lower-frequency component of the input image data, thereby ensuring that the original image is reproduced with the overall image data being compressed while maintaining the medium to higher frequency component.